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Literature Review Methodologies for Post- School Outcomes Follow-Up or 
Follow-Along Studies for Youth with Disabilities 



I ntroduction 

The purpose of this brief is to summarize the methodologies and outcome measures 
used in Post-School Outcomes follow-up and follow-along studies of adolescents with 
disabilities. As a requisite in the Request for Proposal for funding of the National Post- 
School Outcomes Center (NPSO), knowledge development activities were to include: 
"Conducting a literature review on the measures and methodologies that are used to 
collect data on secondary, transition, and postsecondary school outcomes for youth 
with disabilities." (Federal Register, June 4, 2004, p. 31592) This literature review 
was completed with the purpose of providing NPSO and State Education Agencies 
(SEAs) with information for how to conduct follow-up studies in the most efficient and 
scientific exacting manner. 

The brief contains an overview of the (a) methods used to conduct the literature 
review, (b) summary of findings for methodologies of collecting post-school outcomes 
data, and (c) a bibliography of the studies summarized in this review. 

Method 

The purpose of the NPSO Center is to conduct "...research activities and provide 
technical assistance to states, schools, communities, and agencies in developing and 
implementing practical, efficient, cost-effective, and sustainable strategies for 
collecting and using outcome data to improve secondary, transition, and 
postsecondary school outcomes." (Federal Register, J une 4, 2004, p. 31592). In light 
of this purpose, we examined the data collection methods and measures used in 
publications reporting post-school employment, further education, and/or 
independent living data. Specific to Indicator 14, states are required to report the: 



Percent of youth who had lEPs, are no longer in secondary school and who have been 
competitively employed, enrolled in some type of postsecondary school, or both, 
within one year of leaving high school. (20 U.S.C. 1416(a)(3)(B)). 



Requisite to collect Indicator 14 data, we used the following questions to guide our 
review of the methodologies used to collect post-school outcomes data: 

■ What was the source of information (data)? 

■ What was the nature of the sample? 

■ What data collection methods were used? 

■ What predictor and outcome indicators were used in these studies and how 
were they operationalized? 

Completing the literature review consisted of (a) identifying appropriate 
studies, (b) coding the studies, and (c) analyzing the findings. The following sections 
describe these processes. 
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Identifying Appropriate Studies 

We used multiple steps to identify studies (Cooper, 1998). First, bibliographies of 
"classic" longitudinal and follow-up studies were provided to the NPSO Center by the 
Division of Career Development and Transition Research committee and the What 
Works in Transition: Systemic Review Project. Second, NPSO located these studies in 
electronic databases and reviewed the keywords and major descriptors as assigned 
by the national cataloging services (e.g., ERIC, PsychINFO). Third, NPSO staff used 
these common major descriptors as keywords to search electronic databases (e.g., 
ERIC, PsychINFO, and Dissertation Abstracts) to locate additional studies with 
characteristics similar to the "classic" studies. Keywords used were disabilities, 
education outcomes, special education, longitudinal studies, follow-up studies and 
post-school outcomes. Additionally, we combined the original keywords with specific 
disability categories (e.g., SLD and post-school outcomes) to form the search term. 
Fourth, we identified additional studies by using the reference sections of the 
identified publications. In conjunction with the keywords, we applied the following 
search parameters: (a) peer-reviewed journal articles, (b) technical reports, (c) 
dissertations and theses, (d) conference papers and presentations, and (e) date 
range of 1965 to 2004. This broad search yielded more than 500 publications that we 
initially screened for the literature review. 

The titles and abstracts of the prospective studies were carefully reviewed. We used 
four criteria for inclusion in our initial coding process: 

■ Were data reported on at least one post-school outcome domain (i.e., 
further education, employment, or independent living)? 

■ Was the study conducted using participants in the United States? 

■ Were adolescents/young adults with disabilities included as study 
participants? 

■ Were data linked to participants leaving a school system? 

If the above criteria could not be determined from reviewing the abstract, a paper 
copy of the publication was obtained and reviewed for appropriateness. In all, we 
obtained paper copies for 298 publications. We conducted a second review of the full 
article/ report using our same inclusion criteria and reduced the number of 
appropriate studies to 170. 

Coding the Studies 

A comprehensive coding protocol was developed in alignment with our guiding 
methodological questions. The literature review coding protocol was adapted from a 
coding document developed by NPSO staff to code and analyze each state's Annual 
Performance Report for Indicator 14. 

The coding team consisted of the Knowledge Development Associate who coordinated 
the literature review process, a doctoral-level technical assistance (TA) provider, and 
three doctoral students who serve as Knowledge Development Assistants to the 
Center. To establish coder reliability, we followed these steps: First, a coding protocol 
manual was developed operationalizing each of the coding questions and descriptors. 
Second, a coding training was held to review the coding protocol and the operational 
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definitions of the coding document. The coding team individually coded two articles 
and then met to compare codes. Coding discrepancies were discussed. We repeated 
this process until 90% coding reliability was gained on key coding questions across 
the team. Third, a consensus coding process was established. All studies were 
randomly assigned to the three doctoral students and TA provider. Each study was 
then independently coded. Upon completion of coding each study, the coder 
completed a confidence rating of the accuracy of the coding. When a low confidence 
rating was obtained, the study was assigned to a second coder for review. The first 
and second coder then met and discussed codes until consensus was met. When the 
two coders could not reach consensus, the Knowledge Development Associate 
reviewed the coding items in question, and made the final decision about the code to 
be assigned. 

During this coding process, we reduced the number of articles coded for two reasons. 
First, multiple articles were identified that were written from one study (e.g., the 
National Longitudinal Transition Study— NLTS2); in these circumstances we identified 
the article or report that best described the methods used in the study and coded the 
study; therefore only one publication was retained for the literature review even 
though multiple publications were written from the study. When it was not clear 
whether studies shared the same data source or methodology, we retained both 
studies in the literature review. Second, upon closer review of the identified study, 
studies that did not meet our initial criteria were excluded. In all, we coded and 
entered 98 publications into an electronic database created for this project. 

Analyzing the Findings 

Upon entering the coding protocol into the database, the data were extracted and 
validated for discrepancies in data format. In addition, categories that had been 
coded as "other" were reviewed and collapsed, when appropriate, into existing 
descriptors or new code categories. Simple descriptive statistics were then calculated 
and summarized as follows. 

Summary of Findings 

The following summarizes the guiding questions used to identify the methodologies in 
prior studies of post-school outcomes of adolescents and young adults with 
disabilities. In most cases, we report the five most frequently occurring descriptors 
for each question. Percentages are based on the total number of publications in the 
review, N = 98. Some questions were designed to allow more than one response 
option. In cases where multiple response options were available, and the total 
number of responses exceeds 98, we do not present percentages. 

Source of I nformation (Data) Used in the Study 

Figure 1 presents the source of information used across the 98 studies. Overall, 
nearly three quarters (73%) used some type of data collection system created 
exclusively for the study. Specifically, 

• 46 (47%) studies utilized a Data collection system, defined as data collected 

solely for the purpose of study. 
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• 25 (26%) studies utilized a Combination of data collection and 
administrative records as the data source. 

• 19 (19%) studies utilized an Administrative records database, defined as 
an existing database used for multiple purposes (e.g., Employment Division, 
Child count database, vocational rehabilitation), as the data source. 

• Four (4%) of the publications utilized some "Other" source of data (e.g., an 
SEA database, program evaluation database, or coordinated follow-up studies). 

• Four (4%) studies had a data source that was Unclear or Missing data for 
this question. 



Figure 1. Data Source 




■ Data collection system 



D Combined data 
collection & 
administrative records 
□ Administrative Records 



□ Other, unclear, or 
missing 



Data Collection Methods 

A variety of data collection methods were used to gather post-school outcomes 
indicators across the 98 publications. As described above, some studies used multiple 
sources of the data (e.g., extant, direct data collection); therefore, the following lists 
summarize the methods applied to the source of the data. The most commonly 
reported methods of data collection are listed below. 

• 88 studies used Survey methodology as a method of data collection. 

• 42 studies used a review of School records as a method of data collection. 

• 9 studies used a review of Extant data as a method of data collection. 

• 3 studies used a review of Student lEPs as a method of data collection. 

Of the studies that used Survey methodologies, the most commonly reported 
types of survey methodologies are listed below. 

• 54 studies indicated the use of Interviews (e.g., phone, or face-to-face) to 
conduct the survey. 

• 20 studies indicated the use of Any combination (phone and mail surveys, or 
face-to-face and mail) to conduct the survey. 

• 8 studies indicated the use of a Mail survey only to conduct the survey. 
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Nature of the Sample 

The goal of a good sampling plan is to obtain a sample representative of the targeted 
population. The following information summarizes the sampling frame used to 
describe the population of interest in the studies reviewed. 

• 33 (34%) studies used Part of a state as the sample and included descriptors 
such as a school district, county, or region in a state. 

• 24 (25%) studies used an Entire state as the sample. 

• 19 (19%) studies used a School or program level as the sample. 

• 10 (10%) used a National sample. 

• 7 (7%) used some "Other" sample or had Missing data for this question. 

• 5 (5%) of the publications utilized a Multiple state sample. 



Predictor Variables 

Demographic characteristics/ variables were reported in 83 of the 98 (84.7%) 
publications. Typically multiple demographic characteristics were reported. It must be 
noted that in most studies the sample sizes were not large enough to use these 
predictor variables with confidence in measuring differences in post-school outcomes 
within sub-populations (e.g., disability type). The most commonly used variables are 
reported below. 



. 64(77.1%) 

. 36(43.3%) 

. 14(16.9%) 

• 12 (14.5%) 

. 12 (14.5%) 

variable. 



studies used Disability type as a demographic variable, 
studies used Sex as a demographic variable, 
studies used Race/ ethnicity as a demographic variable, 
studies used Geographic location as a demographic variable, 
studies used Dropout/early leaver status as a demographic 



Transition program service variables were also used as predictor variables in some 
studies. These variables related to the type of program and/or services individuals 
received while in high school. Program service predictors were used in 42 of the 98 
(42.9%) studies. The most commonly used transition program service predictors are 
reported below. 

• 17 (17%) studies used Vocational training, coursework, or exploration as 

a transition program service predictor. 

• 16 (16%) studies used Competitive employment while in high school as a 

transition program service predictor. 

• 10 (10%) studies used Other (e.g., educational placement, hours in special 
education, services received in high school, transition planning instituted 
beginning in 1990, level of service, used community resources, autonomy, 
peer influence) as a transition program service predictor. 

• 9 (9%) studies used Type of program or classes in high school as a 
transition program service predictor. 

• 6 (6%) of the publications used Completion document as a transition 
program service predictor. 
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Post-School Outcomes 

Measures for post-school 
outcomes were classified typically 
by the transition domains: 
employment, post-secondary 
education, and independent/ 
quality of life. Figure 2 presents 
the percent of studies reporting 
outcomes in each of these broad 
transition domains. Operational 
definitions for each transition 
domain were diverse. No 
consistent definitions of the three 
constructs were identified; 
therefore, we provide no summary 
of the operational definitions. Overall, employment outcomes were measured in 
almost all studies (94%), followed by post- secondary education outcomes (69%), 
and then independent living outcomes (64%). A summary of the types of variables 
for each broad transition domain is given below. 

Employment Outcomes 

Across the studies, multiple variables were used to measure the employment status 
of youth with disabilities. Ninety-two of the 98 articles used some type of 
employment- related measure. Employment outcome indicators were categorized 
around the following seven employment- related categories: 

• I ncome earned or other financial questions, (e.g., earnings reported by 
yearly, quarterly, monthly, weekly, hourly, and piece rate); 

• Type or kind of jobs held, (e.g., competitive, sheltered, retail, food services, 
etc.); 

• Employment status, (e.g., paid/unpaid, engaged/unengaged, volunteer 
services, and unemployment status); 

• Number of hours worked, (e.g., full- or part-time, and seasonal); 

• Types of Benefits, (e.g., vacation, retirement, sick leave, and major 
medical/dental insurance); 

• J ob stability, (e.g., length of time on job, number of jobs, advancements, 
fired, laid off, and quit); 

• Other or miscellaneous, (e.g., job satisfaction, military, and how a job was 
obtained, etc.). 

Post-Secondary Education Outcomes 

Again, multiple variables were used to measure post- secondary education status of 
youth with disabilities. Sixty-eight of the 98 studies used some type of post- 
secondary education measure. Post- secondary indicators were categorized around 
the following post- secondary education- related categories: 



Figure 2. Type of Post- School Outcomes 
Used per Study 




Employment Education Independent 

Living 
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• Kind of institutions attended, (e.g., 2- and 4-year institutes, 
vocational/trade schools, community colleges, and all public institutes of higher 
education); 

• Enrollment rates, (e.g., engagement, years attended, and training); 

• Degree/ certificate completion rates (e.g., type degrees sought, and field 
of study); 

• Other, (e.g., completion of GED, types of classes taken, and enrolled, but did 
not pass any classes, etc.). 

Independent Living Outcomes 

Sixty-three of 98 studies used some type of independent living or quality of life 
measure. Independent living outcome indicators were categorized around the 
following broad categories: 

• Living status, (e.g., on own, roommates, family, supervised living, military, 
dorm, etc.); 

• Community involvement, (e.g., voting, transportation, etc.); 

• Social relationships, (e.g., leisure activities); 

• Assistance, (e. g., public assistance, accessing services, and services 
received); 

• Finances, (e.g., ability to pay bills, family income, etc.); 

• Other, (e.g., legal issues, family status, and satisfaction). 

Lessons Learned 

In summary, lessons learned from this literature review will be targeted to 
methodological considerations relevant to SEAs as they develop a post-school 
outcomes data collection system. Data garnered from the data collection systems can 
then be used to programmatically improve services for youth with disabilities as they 
transition into adulthood, thereby improving life outcomes for this population. The 
following are several themes that SEAs may consider when developing their post- 
school outcomes data collection system. 

Survey Methodology versus Extant Data 

Survey methodology, not the use of extant data, dominated how data were collected 
in the extant literature. Researchers may have made this decision due to the lack of 
access to extant data sources on individuals with disabilities. For example, 
researchers may not have had access to unique student identifiers that could link 
extant data sources post-exit (e.g., the employment division). That being said, 
almost 25% of the studies were able to combine a data collection system with some 
type of extant data. SEAs may want to explore further, or create opportunities, to 
combine these two methods for reporting post-school outcomes data. For example, 
can unique identifiers be established for SEAs to link post-school outcomes data of 
Indicator 14 with Indicators 1 (high school completion), 2 (drop-outs), and 13 
(transition goals) data? By linking these data, analyses can be conducted to link in- 
school transition related programmatic services to performance on post-school 
outcomes. 




In reviewing studies that used survey methodology, interviewing— either face-to-face 
or by telephone— was the most frequently used survey method with former students. 
While this method has been attributed to higher response rates and greater accuracy 
of information, this method of data collection is also resource intensive. Furthermore, 
based on the time frame of the studies completed (1975-2005), web- based survey 
technology was not a method of data collection used in the studies reviewed. With 
these things in mind, SEAs will have multiple options from which to choose when 
determining the most appropriate method of data collection for them. Decisions must 
be made based on the benefits versus the costs of each method of data collection. 

Static Demographic and Programmatic Variables 

In the studies reviewed, demographic variables were commonly reported as 
aggregates within the sample. For example, a researcher may have collapsed several 
disability types into a single larger category as opposed to summarizing every 
disability type. In these studies the sample sizes often were not large enough to 
examine sub-sample populations with statistical confidence, especially for low 
incidence disability categories, thus the reason for aggregating the sample. 
Additionally, in the studies reviewed, programmatic variables were used less often 
than demographic variables, but when programmatic variables were collected, 
analyses, at times, were completed to examine if high school services were predictive 
of specific post- school outcomes. 

The purpose for SEA's use of demographic data is twofold: First, demographic data, 
can be used to determine the representativeness of the sample to the population; 
and second, to identify the student group for whom a given program is most, and 
least, effective. SEAs have been charged with reporting demographic information 
relevant to disaggregated samples based on disability type, sex, and race/ ethnicity. 

It will be important for states to collect the demographic information recommended 
by OSEP and then make careful decisions regarding how to collapse these 
demographic variables for reporting purposes. 

Based on the use of both demographic (sex, ethnicity, disability type) and 
programmatic (vocational services, completion document, etc.) predictor variables in 
the reviewed studies, these variables may also be used by SEAs to examine program 
efficacy. SEAs may be constrained by small sub- sample sizes specifically for 
demographic variables not prevalent in their states (e.g., low incidence disability 
types, race/ ethnicity). However, even with a constrained sample size, effective 
analyses strategies to examine post-school outcomes demonstrated by demographic 
variables will be important to assess the efficacy of services provided to various sub- 
populations. Moreover, as noted before, linking post-school outcomes with the in- 
school programs will be a challenge for SEAs and will require their further 
investigation. SEAs may want to explore strategies for the use of a unique student 
identifier that can transfer to post-school outcomes data collection or use of extant 
data. 

Post-School Outcomes 

Post-school outcomes in the studies reviewed were measured across the three broad 
transition domains: (a) employment, (b) post-secondary education, and (c) 
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independent living/quality of life. All three measures were defined in a variety of 
ways without revealing a central operational definition. Often multiple measures 
within each domain were used to further explore the outcomes of the sample in that 
domain (e.g., employed, type of job, length of time on job, benefits, etc.). 
Employment- related variables represented the most frequently reported transition 
domain measured across these studies, with almost two-thirds of studies also 
including post-secondary education and independent living outcomes. 

Although a combination of competitive employment and post-secondary enrollment is 
the minimum reporting requirement by SEAs to OSEP, SEAs may find it useful to 
collect other types of post-school outcomes data to further examine how former 
students fare in adulthood. For example, independent living/quality of life measures, 
while not a reporting requirement for states, reveal much about the status of former 
students with low incidence disabilities. It is possible that the post-school outcomes 
for former students with moderate and severe disabilities only would be captured 
with the inclusion of this indicator category. Additionally, SEAs may find it useful to 
explore operational definitions used by partner agencies (e.g., vocational 
rehabilitation) within their state to determine a definition that will be meaningful 
across multiple agencies. For example, a common definition of full-time employment, 
defined as at least 35 hours per week, would be especially useful when accessing 
extant data from the State Employment Departments for the purpose of reporting 
post-school outcomes data. 

Summary 

The purpose of this literature review is to identify the methodologies used to gather 
post-school outcomes for individuals with disabilities relevant to employment, post- 
secondary education, and independent living. To that end, the literature review can 
only offer guidance to SEAs for "...collecting data on secondary, transition, and 
postsecondary school outcomes for youth with disabilities" (Federal Register, June 4, 
2004, p. 31592). It is up to State Departments of Education, as they embark on the 
task of collecting, analyzing, reporting, and using post-school outcomes data, to 
examine the optimal practices in light of their resources and decide what information 
will be most beneficial to them as they develop their data collection and reporting 
system. It is our hope that the recommendations from our literature review will be 
considered when making decisions for methods of data collection, choice of 
measures, analyses, and use of post-school outcomes data. 
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